Clean up mk_GEOSldasRestarts and process_rst.csh #551

weiyuan-jiang · 2022-05-10T12:26:34Z

Revise processing of GEOSldas restarts:

Adopt new GMAO_Shared utility remap_restarts.py.
Re-mapping of LDASsa-formatted restarts is no longer supported.
Update ldas_setup to python3.
Replace process_rst.csh with cleaned-up python script process_rst.py.

This branch should have zero-diff except:

The regridded restart has non-zero TSURF.
The regridded restart has the same set of variables as the original restart, i.e. , WW, FR, CQ, CH etc. are not added if the original restart does not have them.
This branch depends on newer versions of GEOSgcm_GridComp and MAPL.

…Rs being merged in GMAO_Shared and GEOSgcm_GridComp

gmao-rreichle · 2022-07-21T13:20:53Z

@weiyuan-jiang, @biljanaorescanin,
Now that Scott has merged GEOSgcm_GridComp PR#571 and
is in the process of merging GMAO_Shared PR#238, I'm wondering if we should merge this GEOSldas PR as is or tweak it further.
A quick inspection of the changes suggests that this PR calls mk_catchANDcnRestarts.F90 directly, rather than going through the new remap_catchANDcn.py. Would it make sense to further modify this PR to use the new python script?
In this context, I know Scott needs command line options for the new remap restarts python script(s) before he can use it in the GCM setup script. This means we'll have to do some additional work on the python scripts, and it may be better to wait until this new work is complete before we convert the LDAS setup to use the new remap restarts utilities. (In a nutshell, Scott needs a one-line command option to remap restarts, which isn't an option right now because everything goes through yaml files. I can't say for sure, but I don't think we need to abandon the yaml files. Rather, it should be possible to create a wrapper script or function that deals with creating the yaml file when given command line inputs.)
Thoughts?
PS: Has this PR been tested recently? I don't see a test report in the Conversation. Before we merge the PR, we should probably test it again after Scott completed his merger of GMAO_Shared PR#238.

weiyuan-jiang · 2022-07-21T14:03:56Z

I tested this branch before but I will test it again after things are settled. Yes, I will add a command line to generate the yaml file.

weiyuan-jiang · 2022-08-10T23:25:53Z

This PR ( may need to change component.yaml ) is zero diff except global assim run. This assim run used LDASsa's restart, which is not supported any more. We should change the restart.

This reverts commit b3aeb0e.

…p for merging branch into GEOSldas "develop"

gmao-rreichle · 2022-08-26T20:54:00Z

The two most recent commits revert the components.yaml file back to the "develop" version, in prep for merging the PR into "develop".
This PR requires first merging of GEOS-ESM/GMAO_Shared#276

gmao-rreichle · 2022-08-30T12:59:49Z

src/Applications/LDAS_App/ldas_setup

+                self.in_rstfile  = '/discover/nobackup/projects/gmao/ssd/land/l_data/LandRestarts_for_Regridding' \
+                                  '/CatchCN/M09/20151231/catchcn_internal_rst' 
+                self.in_tilefile = '/discover/nobackup/ltakacs/bcs/Heracles-NL/SMAP_EASEv2_M09/SMAP_EASEv2_M09_3856x1624.til'
+              elif (self.catch == 'catchcnclm45'):


@weiyuan-jiang, @gmao-jkolassa:
If I'm guessing right, the current ldas_setup does not have information about a dummy restart file for CatchCNCLM45. It looks like the dummy restart files for the other model versions are all in the M09 tile space. @gmao-jkolassa, do you have an M09 restart for CatchCNCLM45 that could be used as a dummy restart here? If not, we could use the M36 restart from the LDAS_GLOBALCNCLM45 nightly test.
@weiyuan-jiang, please work with @gmao-jkolassa to implement a dummy CatchCNCLM45 restart in ldas_setup.

I do not have an M09 restart file readily available, but I can generate one if that is preferred. Please let me know.

@weiyuan-jiang I will send you the path to an M36 restart that can be used if an M09 restart is not needed.

@gmao-jkolassa: I'll leave it up to you to decide if we need an M09 dummy restart or if the M36 restart from the nightly test case is sufficient. Note that this is just a dummy and meant to give users a start by providing a properly formatted restart. Users will need to do some spin-up (unless the dummy restart itself is spun up properly, in the same tile space, and for the desired restart date, which is not likely to ever happen).

Since this is only meant to serve as a dummy restart with the understanding that some additional spin-up is necessary, I think it is fine to use the M36 restart file from the nightly test. I will work with @weiyuan-jiang to make sure it is copied to an accessible space.

src/Applications/LDAS_App/process_rst.py

gmao-rreichle · 2022-08-31T19:57:32Z

PRs #569 and #574 should be merged first to minimize conflicts.
Prior to merging the present PR, need to change CMakeLists.txt in the present branch to make it compatible with changes in #569.

…d conflict in src/Applications/LDAS_App/CMakeLists.txt

src/Applications/LDAS_App/ldas_setup

gmao-rreichle · 2022-09-01T19:31:31Z

PRs #569 and #574 should be merged first to minimize conflicts. Prior to merging the present PR, need to change CMakeLists.txt in the present branch to make it compatible with changes in #569.

PRs #569 and #574 have been merged, and the present branch has been updated to "develop," incl. resolution of the conflict in CMakeLists.txt.

Successfully completed LDAS Nightly 0-diff tests (Intel only) after e6994a3 (i.e., before updating of CatchCN dummy restart files).

biljanaorescanin · 2022-09-01T22:41:30Z

@gmao-rreichle all intel tests passed

biljanaorescanin · 2022-09-02T10:58:34Z

@gmao-rreichle Since this is on develop and current develop fails after change we did for new GNU compiler, these tests fail for same tests. All intel tests passed.

gnuconus pass pass 12 min pass/FAIL -- / --
gnuglobal -- -- -- pass/FAIL pass/FAIL
gnuglobalcs -- -- -- pass/FAIL pass/FAIL
gnuglobalcnclm4 -- -- -- pass/FAIL -- / --
gnuglobalcnclm45 -- -- -- pass/FAIL -- / --

This branch is using mk_catchANDcnRestarts.x

e72cf69

weiyuan-jiang added 0-diff cleanup Contingent -- Do Not Approve labels May 10, 2022

weiyuan-jiang requested a review from a team as a code owner May 10, 2022 12:26

weiyuan-jiang and others added 4 commits May 11, 2022 11:51

change components for testing purpose

d76fbab

set in_tilefile as string by default

cc4cda2

reverting components.yaml back to develop branch in anticipation of P…

f2909f2

…Rs being merged in GMAO_Shared and GEOSgcm_GridComp

Merge branch 'develop' into feature/wjiang/clean_mk_restarts

abe3d18

gmao-rreichle marked this pull request as draft July 21, 2022 13:10

weiyuan-jiang added 6 commits August 8, 2022 09:16

changed to python 3 and use remap_restarts

347a88d

change the script for processing restart

120d38f

bug fix

0a338cd

fix glob

2a3af9c

correct python3 division

e7d2fa0

change chmod

795cf09

weiyuan-jiang added 3 commits August 11, 2022 22:50

rm support for LDASsa restart. rm process_rst.csh

3df4bc2

search vegrst

2640504

change componet for test

b3aeb0e

biljanaorescanin mentioned this pull request Aug 19, 2022

Add Option to Use MODIS-based Snow Albedo Climatology GEOS-ESM/GEOSgcm_GridComp#618

Merged

gmao-rreichle added 2 commits August 26, 2022 16:47

Revert "change componet for test"

45ec4c0

This reverts commit b3aeb0e.

reverting GMAO_Shared back to "main" branch in components.yaml in pre…

fa7f594

…p for merging branch into GEOSldas "develop"

fix typos in comments (ldas_setup)

1f51c48

gmao-rreichle reviewed Aug 30, 2022

View reviewed changes

src/Applications/LDAS_App/process_rst.py Show resolved Hide resolved

extend FP restart dates

f8828a7

gmao-rreichle and others added 3 commits September 1, 2022 13:38

resolved merge conflict in CMakeLists.txt

e3c52a6

Merge branch 'develop' into feature/wjiang/clean_mk_restarts. Resolve…

e6994a3

…d conflict in src/Applications/LDAS_App/CMakeLists.txt

update catchcn restart dummy files

17901e5

gmao-rreichle reviewed Sep 1, 2022

View reviewed changes

src/Applications/LDAS_App/ldas_setup Show resolved Hide resolved

fix catchcn45 tile file

4c85309

biljanaorescanin added 2 commits September 1, 2022 15:41

missing else

664049a

no need for else

f93bb94

else syntax error

3379fc0

gmao-rreichle removed the Contingent -- Do Not Approve label Sep 2, 2022

gmao-rreichle marked this pull request as ready for review September 2, 2022 13:27

gmao-rreichle requested a review from a team as a code owner September 2, 2022 13:27

gmao-rreichle approved these changes Sep 2, 2022

View reviewed changes

gmao-rreichle merged commit 34d08fd into develop Sep 2, 2022

gmao-rreichle deleted the feature/wjiang/clean_mk_restarts branch September 2, 2022 13:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clean up mk_GEOSldasRestarts and process_rst.csh #551

Clean up mk_GEOSldasRestarts and process_rst.csh #551

weiyuan-jiang commented May 10, 2022 •

edited by gmao-rreichle

Loading

gmao-rreichle commented Jul 21, 2022 •

edited

Loading

weiyuan-jiang commented Jul 21, 2022

weiyuan-jiang commented Aug 10, 2022

gmao-rreichle commented Aug 26, 2022

gmao-rreichle Aug 30, 2022

gmao-jkolassa Aug 30, 2022

gmao-rreichle Aug 30, 2022

gmao-jkolassa Aug 30, 2022

gmao-rreichle commented Aug 31, 2022

gmao-rreichle commented Sep 1, 2022

biljanaorescanin commented Sep 1, 2022

biljanaorescanin commented Sep 2, 2022 •

edited

Loading

Clean up mk_GEOSldasRestarts and process_rst.csh #551

Clean up mk_GEOSldasRestarts and process_rst.csh #551

Conversation

weiyuan-jiang commented May 10, 2022 • edited by gmao-rreichle Loading

gmao-rreichle commented Jul 21, 2022 • edited Loading

weiyuan-jiang commented Jul 21, 2022

weiyuan-jiang commented Aug 10, 2022

gmao-rreichle commented Aug 26, 2022

gmao-rreichle Aug 30, 2022

Choose a reason for hiding this comment

gmao-jkolassa Aug 30, 2022

Choose a reason for hiding this comment

gmao-rreichle Aug 30, 2022

Choose a reason for hiding this comment

gmao-jkolassa Aug 30, 2022

Choose a reason for hiding this comment

gmao-rreichle commented Aug 31, 2022

gmao-rreichle commented Sep 1, 2022

biljanaorescanin commented Sep 1, 2022

biljanaorescanin commented Sep 2, 2022 • edited Loading

weiyuan-jiang commented May 10, 2022 •

edited by gmao-rreichle

Loading

gmao-rreichle commented Jul 21, 2022 •

edited

Loading

biljanaorescanin commented Sep 2, 2022 •

edited

Loading